Cross-environment application compatibility

ABSTRACT

A seamless cross-environment workflow is provided in a multi-operating system computing environment. The multi-operating system computing environment may include a mobile operating system and a desktop operating system running concurrently and independently on a mobile computing device. Two or more application programs, running in independent operating systems, share user interaction state information including user data, user settings, and/or application context information. Interaction state information may be shared for applications that are used primarily to access and edit local user content as well as applications that communicate to a remote server or access and navigate other remote content (e.g., Internet-based application, browser, etc.). The mobile computing device may be a smartphone running the Android mobile operating system and a full desktop Linux distribution on a modified Android kernel.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Application is a non-provisional of and claims the benefit of thefiling date of U.S. Provisional Application Nos. 61/389,117, filed Oct.1, 2010, entitled “Multi-Operating System Portable Docking Device”;61/507,199, filed Jul. 13, 2011, entitled “Dockable Mobile SoftwareArchitecture”; 61/507,201, filed Jul. 13, 2011, entitled“Cross-Environment Communication Framework”; 61/507,203, filed Jul. 13,2011, entitled “Multi-Operating System”; 61/507,206, filed Jul. 13,2011, entitled “Auto-Configuration of a Docked System in a Multi-OSEnvironment”; and 61/507,209, filed Jul. 13, 2011, entitled “Auto-Wakingof a Suspended Secondary OS in a Dockable System,” wherein the entirecontents of the foregoing priority applications are incorporated hereinby reference for all purposes.

BACKGROUND

1. Field

This Application relates generally to the field of mobile computingenvironments, and more particularly to supporting multiple userenvironments through the use of multiple operating systems in a singlemobile computing device.

2. Relevant Background

Mobile computing devices are becoming ubiquitous in today's society. Forexample, as of the end of 2008, 90 percent of Americans had a mobilewireless device. At the same time, the capabilities of mobile devicesare advancing rapidly, including smartphones that integrate advancedcomputing capabilities with mobile telephony capabilities. Mobileproviders have launched hundreds of new smartphones in the last threeyears based upon several different platforms (e.g., Apple iPhone,Android, BlackBerry, Palm, and Windows Mobile). In the U.S., smartphonepenetration reached almost 23% by the middle of 2010, and over 35% insome age-groups. In Europe, the smartphone market grew by 41% from 2009to 2010, with over 60 million smartphone subscribers as of July 2010 inthe five largest European countries alone.

While smartphones are gaining in popularity and computing capability,they provide a limited user experience. Specifically, they typicallyhave an operating system that is modified for mobile device hardware anda restricted set of applications that are available for the modifiedoperating system. For example, many smartphones run Google's Androidoperating system. Android runs only applications that are specificallydeveloped to run within a Java-based virtual machine runtimeenvironment. In addition, while Android is based on a modified Linuxkernel, it uses different standard C libraries, system managers, andservices than Linux. Accordingly, applications written for Linux do notrun on Android without modification or porting. Similarly, Apple'siPhone uses the iOS mobile operating system. Again, while iOS is derivedfrom Mac OS X, applications developed for OS X do not run on iOS.Therefore, while many applications are available for mobile operatingsystems such as Android and iOS, many other common applications fordesktop operating systems such as Linux and Mac OS X are not availableon the mobile platforms.

Accordingly, smartphones are typically suited for a limited set of userexperiences and provide applications designed primarily for the mobileenvironment. In particular, smartphones do not provide a suitabledesktop user experience, nor do they run most common desktopapplications. As a result, many users carry and use multiple computingdevices including a smartphone, laptop, and/or tablet computer. In thisinstance, each device has its own CPU, memory, file storage, andoperating system.

Connectivity and file sharing between smartphones and other computingdevices involves linking one device (e.g., smartphone, running a mobileOS) to a second, wholly disparate device (e.g., notebook, desktop, ortablet running a desktop OS), through a wireless or wired connection.Information is shared across devices by synchronizing data betweenapplications running separately on each device. This process, typicallycalled “synching,” is cumbersome and generally requires activemanagement by the user.

SUMMARY

Embodiments of the present invention are directed to providing themobile computing experience of a smartphone and the appropriate userexperience of a secondary terminal environment in a single mobilecomputing device. A secondary terminal environment may be somecombination of visual rendering devices (e.g., monitor or display),input devices (e.g., mouse, touch pad, touch-screen, keyboard, etc.),and other computing peripherals (e.g., HDD, optical disc drive, memorystick, camera, printer, etc.) connected to the computing device by awired (e.g., USB, Firewire, Thunderbolt, etc.) or wireless (e.g.,Bluetooth, WiFi, etc.) connection. In embodiments, a mobile operatingsystem associated with the user experience of the mobile environment anda desktop operating system associated with the user experience of thesecondary terminal environment are run concurrently and independently ona shared kernel.

According to aspects consistent with various embodiments, across-environment workflow is provided by opening a document file with afirst application running on a first operating system, the user contentof the document file representing a stored document state, acceptinguser input in the first application, the first application having, as aresult of responding to the user input, a first user interaction state,the first user interaction state including a modified document state,storing application state information representing the first userinteraction state within the first computing environment, theapplication state information including application context information,releasing a file lock of the document file by the first applicationwithout saving the modified document state to the document file,opening, by a second application running on a second operating system,the document file, accessing, by the second application, the applicationstate information associated with the first user interaction state, andapplying, by the second application, the application context informationto present an interaction state of the second application that issubstantially similar to the first user interaction state.

According to other aspects consistent with various embodiments, thecross-environment workflow includes opening, by the second application,a temporary document file created by the first application, thetemporary document file representing changes between the stored documentstate and the modified document state. The application contextinformation may include a hint to the temporary file. The firstapplication may be displayed within a first user environment and thesecond application may be displayed within a second user environment.The first and second operating systems may run concurrently on a sharedkernel of a mobile device. The application context information may bestored in a database accessed by the first and second applicationsthrough an application programming interface.

According to other aspects consistent with various embodiments, theapplication context information may include at least one userinteraction state parameter not represented in the document file. Theapplication context information may include information associated witha non-completed user action and/or a current user selection of usercontent information. The opening of the document file by the secondapplication may be in response to detecting a docking event of a mobiledevice.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are illustrated in referencedfigures of the drawings, in which like numbers refer to like elementsthroughout the description of the figures.

FIG. 1 illustrates a computing environment that provides multiple usercomputing experiences, according to various embodiments.

FIG. 2 illustrates an exemplary system architecture for a mobilecomputing device, according to various embodiments.

FIG. 3 illustrates an operating system architecture for a computingenvironment, according to various embodiments.

FIG. 4 illustrates a mobile operating system architecture that may beemployed in a multiple operating system computing environment, accordingto various embodiments.

FIG. 5 illustrates aspects of an operating system architecture for acomputing environment, according to various embodiments.

FIG. 6 illustrates a mobile operating system architecture that may beemployed in a multiple operating system computing environment, accordingto various embodiments.

FIG. 7 illustrates a multiple operating system computing environment forproviding translation of application data, user settings, and/orapplication context across applications running in separate operatingsystems, according to various embodiments.

FIG. 8 illustrates a computing environment configuration, according tovarious embodiments.

FIG. 9 illustrates a computing environment configuration in which amobile computing device is docked with a secondary terminal environment,according to various embodiments.

FIG. 10 9 illustrates a computing environment configuration that showshow aspects of the computing environment of FIG. 7 provide a seamlessworkflow across multiple applications running on separate operatingsystems, according to various embodiments.

FIG. 11 illustrates an exemplary process flow for translation ofapplication data between separate applications running on independentoperating systems and displayed in separate user environments, accordingto various embodiments.

FIG. 12 illustrates another exemplary process flow for translation ofapplication data between separate applications running on independentoperating systems and displayed in separate user environments, accordingto various embodiments.

DETAILED DESCRIPTION

Mobile telephony devices, (i.e., smartphones, handsets, mobile stations,portable communication devices, etc.) that include computingcapabilities are increasing in popularity. Many of these smartphonesinclude a mobile operating system (“OS”) running on a mobile processor.While mobile processors and mobile OS's have increased the capabilitiesof these devices, smartphones have not tended to replace personalcomputer (“PC”) environments (i.e., Windows, Mac OS X, Linux) such asdesktop or notebook computers at least because of the limited userexperience provided. In particular, the user interface device(s) foundon smartphones are typically tailored to the mobile environment. Forexample, smartphones typically use a small thumb-style QWERTY keyboard,touch-screen display, click-wheel, and/or scroll-wheel as user interfacedevices. Mobile OSs, as well as applications (i.e., “Apps”) developedfor mobile OSs, are typically designed for the constraints of the mobileenvironment including a mobile processor and the user interfacedevice(s) present on mobile devices. Therefore, many applications thathave been developed for PC operating systems are not available formobile OSs (i.e., are not compiled for and do not run on mobile OSs). Inaddition, for some tasks such as typing or editing documents, afull-size keyboard and large display are easier to use than the userinterface components typically found on a smartphone.

Accordingly, users typically use separate computing devices for eachcomputing experience, including a smartphone, tablet computer, laptopcomputer, and/or desktop computer. In this instance, each device has itsown CPU, memory, file storage, and OS. Connectivity and file sharingbetween smartphones and other devices involves linking one device (e.g.,smartphone, running a mobile OS) to a second, wholly disparate device(e.g., notebook, desktop, or tablet running a desktop OS), through awireless or wired connection. Information is shared across devices bysynchronizing data between applications running separately on eachdevice. This process, typically called “synching,” is cumbersome andgenerally requires active management by the user.

FIG. 1 illustrates a computing environment 100 that provides multipleuser computing experiences with a mobile device that includes multipleoperating systems associated with separate user interaction spaces(i.e., user environments), according to various embodiments. A firstuser interaction space 115 of computing environment 100 includesdisplay(s) 116 and I/O devices 118 of mobile computing device 110. Whenmobile computing device 110 is operated as a stand-alone mobile device,mobile OS 130 presents a typical mobile computing user experiencethrough user interaction space 115. The mobile computing experienceprovided by mobile OS 130 typically includes mobile telephonycapabilities and a graphical user interface (“GUI”) suited to userinteraction space 115 including display(s) 116 and I/O device(s) 118.For example, display(s) 116 may be a touch-screen display(s) andapplication programs (i.e., “Apps”) running on mobile OS 130 may becontrolled primarily through a gesture-based GUI of mobile OS 130 usingtouch-screen display(s) 116.

In computing environment 100, mobile computing device 110 may be dockedwith secondary terminal environment 140 that includes I/O devices 144,146, and/or 148. In embodiments, mobile computing device 110 is dockedwith secondary terminal environment 140 by connecting port 120 of mobilecomputing device 110 to port 142 of secondary terminal environment 140.In this instance, secondary terminal environment 140 presents a seconduser interaction space of computing environment 100. In some instances,the second user interaction space may be more suited to a desktopcomputing experience. In these instances, desktop OS 160 can beassociated with secondary terminal environment 140 to provide the fullcapabilities of a notebook, tablet, or desktop computer environmentthrough the second user interaction space.

In embodiments, mobile OS 130 and desktop OS 160 run concurrently on ashared kernel on a processor of mobile computing device 110. Concurrentexecution of a mobile OS and a desktop OS on a shared kernel isdescribed in more detail in U.S. patent application Ser. No. 13/217,108,filed Aug. 24, 2011, entitled “MULTI-OPERATING SYSTEM,” hereinincorporated by reference. In this way, a single mobile computing devicecan provide a mobile computing experience through a first userinteraction space and a desktop computing experience through a seconduser interaction space. While the ability to carry one mobile devicethat can execute multiple operating systems concurrently throughseparate user interaction spaces solves a number of problems for a user,each user interaction space (through the concurrently running mobile OSand desktop OS) generally provides a separate set of availableapplications and user functionality.

Embodiments of the invention are directed to providing a seamlesscross-environment workflow using multiple applications providing similarfunctionality running on separate, independent operating systems.Embodiments facilitate multiple applications sharing a single apparentuser interaction state. In embodiments, a user can switch seamlesslyfrom interacting with user content with an application running on afirst OS (e.g., mobile OS 130) to interacting with the same user contentwith an application running on a second OS (e.g., desktop OS 160).Notably, application context information of the current user interactionstate is maintained when the user switches to the second application,including application context information not typically stored as partof a document file for the applications.

Consider, for example, that a user is editing a document using a firstword processing application running on a first OS through a first userenvironment. The user wishes to switch to editing the same documentthrough a second user environment using a similar word processingapplication that runs on the second OS associated with the second userenvironment. For example, the user may have docked a mobile computingdevice with a second user environment and wishes to switch to editingthe document with the second user environment. The user desires that thetransition occur seamlessly, so they can pick up using the secondapplication exactly where they left off with the first application,including any non-saved data and incomplete user actions. Handling thisscenario involves managing user document files and application stateinformation in a way that applications running in separate processes onseparate operating systems present a single apparent application userinteraction state.

Accordingly, embodiments provide various novel techniques for managingapplication state information and document files of multipleapplications running on multiple separate operating systems. Theembodiments include techniques for managing application stateinformation for applications that primarily operate on local usercontent, client-server applications, and Internet-based applications.

As described above, computing environment 100 provides multiple usercomputing experiences through multiple user interaction spacesassociated with a mobile device running multiple operating systemsconcurrently. Specifically, because mobile computing device 110 includesmultiple OSs, where each OS is suited to a particular computingenvironment, mobile computing device 110 may be adapted with externalI/O devices to provide a broad range of user experiences with a singlemobile computing device. For example, a user may have a mobile computingdevice 110 and a secondary terminal environment 140 that includes akeyboard, display, and/or pointing device(s) in a laptop-type enclosure.When mobile computing device 110 is docked with this laptop-likesecondary terminal environment, the full capabilities of desktop OS 160are available through the secondary terminal environment 140.

FIG. 2 illustrates an exemplary hardware system architecture for mobilecomputing device 110, according to various embodiments. Mobile computingdevice hardware 112 includes mobile processor 114 that includes one ormore CPU cores 204 and external display interface 220. Generally, mobilecomputing device hardware 112 also includes I/O devices 118, memory 206,storage devices 208, touch-screen display controller 210 connected totouch-screen display 116, power management IC 214 connected to battery216, cellular modem 218, communication devices 222, and/or other devices224 that are connected to processor 114 through various communicationsignals and interfaces. I/O devices 118 generally includes buttons andother user interface components that may be employed in mobile computingdevice 110. For example, I/O devices 118 may include a set of buttons,(e.g., back, menu, home, search, etc.), off-screen gesture area,click-wheel, scroll-wheel, QWERTY keyboard, etc. Other devices 224 mayinclude, for example, GPS devices, LAN connectivity, microphones,speakers, cameras, accelerometers, and/or MS/MMC/SD/SDIO cardinterfaces. External display interface 220 may be any suitable displayinterface (e.g., VGA, DVI, HDMI, etc.).

Processor 114 may be an ARM-based mobile processor. In embodiments,mobile processor 114 is a mobile ARM-based processor such as TexasInstruments OMAP3430, Marvell PXA320, Freescale iMX51, or QualcommQSD8650/8250. However, mobile processor 114 may be another suitableARM-based mobile processor or processor based on other processorarchitectures such as, for example, x86-based processor architectures orother RISC-based processor architectures.

While FIG. 2 illustrates one exemplary hardware implementation 112 formobile computing device 110, other architectures are contemplated aswithin the scope of the invention.

For example, various components illustrated in FIG. 2 as external tomobile processor 114 may be integrated into mobile processor 114.Optionally, external display interface 220, shown in FIG. 2 asintegrated into mobile processor 114, may be external to mobileprocessor 114. Additionally, other computer architectures employing asystem bus, discrete graphics processor, and/or other architecturalvariations are suitable for employing aspects of the present invention.

FIG. 3 illustrates OS architecture 300 that may be employed to runmobile OS 130 and desktop OS 160 concurrently on mobile computing device110, according to various embodiments. As illustrated in FIG. 3, mobileOS 130 and desktop OS 160 are independent operating systems.Specifically, mobile OS 130 and desktop OS 160 may have independent andincompatible user libraries, graphics systems, and/or framework layers.Functions and instructions for OS architecture 300 may be stored ascomputer program code on a tangible computer readable medium of mobilecomputing device 110. For example, instructions for OS architecture 300may be stored in storage device(s) 208 of mobile computing devicehardware 112.

In OS architecture 300, mobile OS 130 and desktop OS 160 runconcurrently on shared kernel 320. This means that mobile OS 130 anddesktop OS 160 are running on shared kernel 320 at the same time.Specifically, mobile OS 130 and desktop OS 160 both interface to sharedkernel 320 through the same kernel interface 322, for example, by makingsystem calls to shared kernel 320. Shared kernel 320 manages taskscheduling for processes of both mobile OS 130 and desktop OS 160. Inthis regard, mobile OS 130 and desktop OS 160 are running independentlyand concurrently on shared kernel 320. In addition, shared kernel 320runs directly on mobile processor 114 of mobile computing devicehardware 112, as illustrated by hardware interface 312. Specifically,shared kernel 320 directly manages the computing resources of mobilecomputing device hardware 112 such as CPU scheduling, memory access, andI/O. In this regard, hardware resources are not virtualized, meaningthat mobile OS 130 and desktop OS 160 make system calls through kernelinterface 322 without virtualized memory or I/O access.

As illustrated in FIG. 3, mobile OS 130 has libraries layer 330,application framework layer 340, and application layer 350. In mobile OS130, applications 352 and 354 run in application layer 350 supported byapplication framework layer 340 of mobile OS 130. Application frameworklayer 340 includes manager(s) 342 and service(s) 344 that are used byapplications running on mobile OS 130. For example, applicationframework layer 340 may include a window manager, activity manager,package manager, resource manager, telephony manager, gesturecontroller, and/or other managers and services for the mobileenvironment. Application framework layer 340 may include a mobileapplication runtime environment that executes applications developed formobile OS 130. The mobile application runtime environment may beoptimized for mobile computing resources such as lower processing powerand/or limited memory space. The mobile application runtime environmentmay rely on the kernel for process isolation, memory management, andthreading support. Libraries layer 330 includes user libraries 332 thatimplement common functions such as I/O and string manipulation, graphicsfunctions, database capabilities, communication capabilities, and/orother functions and capabilities.

As illustrated in FIG. 3, desktop OS 160 has libraries layer 360,framework layer 370, and application layer 380. In desktop OS 160,applications 382 and 384 run in application layer 380 supported byapplication framework layer 370 of desktop OS 160. Application frameworklayer 370 includes manager(s) 372 and service(s) 374 that are used byapplications running on desktop OS 160. For example, applicationframework layer 370 may include a window manager, activity manager,package manager, resource manager, and/or other managers and servicescommon to a desktop environment. Libraries layer 360 may include userlibraries 362 that implement common functions such as I/O and stringmanipulation, graphics functions, database capabilities, communicationcapabilities, and/or other functions and capabilities.

In various embodiments of the present disclosure, desktop OS 160 runs ina separate execution environment from mobile OS 130. For example, mobileOS 130 may run in a root execution environment and desktop OS 160 mayrun in a secondary execution environment established under the rootexecution environment. Processes and applications running on mobile OS130 access user libraries 332, manager(s) 342 and service(s) 344 in theroot execution environment. Processes and applications running ondesktop OS 160 access user libraries 362, manager(s) 372 and service(s)374 in the secondary execution environment.

In embodiments, mobile OS 130 and desktop 160 are independent operatingsystems with incompatible user libraries, graphics systems, and/orapplication frameworks. Therefore, applications developed for mobile OS130 may not run directly on desktop OS 160, and applications developedfor desktop OS 160 may not run directly on mobile OS 130. For example,application 352, running in application layer 350 of mobile OS 130, maybe incompatible with desktop OS 160, meaning that application 352 couldnot run on desktop OS 160. Specifically, application 352 may depend onmanager(s) 342, service(s) 344, and/or libraries 332 of mobile OS 130that are either not available or not compatible with manager(s) 372,service(s) 374, and/or libraries 362 of desktop OS 160.

As a result, mobile OS 130 and desktop OS 160 may have different sets ofavailable applications. In this regard, mobile OS 130 and desktop OS 160of OS architecture 300 provide separate user experiences throughseparate sets of applications accessible through separate userinteraction spaces. The user may access the applications available on(i.e., compiled for and loaded within the execution environment of)mobile OS 130 through a first user interaction space associated withmobile OS 130, and the applications available on desktop OS 160 througha second user interaction space associated with desktop OS 160.

As described above, mobile operating systems typically do not use thesame graphics environment as desktop operating systems. Graphicsenvironments for desktop OS's were designed for flexibility and highperformance. For example, the X-window system, used by some desktop OSs,provides platform and network independence at the expense of greaterprocessing and system resources. In contrast, graphics environments formobile OSs are designed more for efficiency and the specific user inputdevices of a mobile computing environment and less for flexibility.Because the graphics environments of mobile and desktop OS's are oftendifferent, an application running on a mobile OS may not be re-directedto display within a user space of a desktop OS by re-directing thegraphics information from the graphics server of the mobile OS to thegraphics server of the desktop OS.

The most widely adopted mobile OS is Google's Android. While Android isbased on Linux, it includes modifications to the kernel and other OSlayers for the mobile environment and mobile processors. In particular,while the Linux kernel is designed for a PC (i.e., x86) CPUarchitecture, the Android kernel is modified for ARM-based mobileprocessors. Android device drivers are also particularly tailored fordevices typically present in a mobile hardware architecture includingtouch-screens, mobile connectivity (GSM/EDGE, CDMA, Wi-Fi, etc.),battery management, GPS, accelerometers, and camera modules, among otherdevices. In addition, Android does not have a native X Window System nordoes it support the full set of standard GNU libraries, and this makesit difficult to port existing GNU/Linux applications or libraries toAndroid.

Apple's iOS operating system (run on the iPhone) and Microsoft's WindowsPhone 7 are similarly modified for the mobile environment and mobilehardware architecture. For example, while iOS is derived from the Mac OSX desktop OS, common Mac OS X applications do not run natively on iOS.Specifically, iOS applications are developed through a standarddeveloper's kit (“SDK”) to run within the “Cocoa Touch” runtimeenvironment of iOS, which provides basic application infrastructure andsupport for key iOS features such as touch-based input, pushnotifications, and system services. Therefore, applications written forMac OS X do not run on iOS without porting. In addition, it may bedifficult to port Mac OS X applications to iOS because of differencesbetween user libraries and/or application framework layers of the twoOSs, and/or differences in system resources of the mobile and desktophardware.

In one embodiment consistent with OS architecture 300, an Android mobileOS and a full Linux OS run independently and concurrently on a modifiedAndroid kernel. In this embodiment, the Android OS may be a modifiedAndroid distribution while the Linux OS (“Hydroid”) may be a modifiedDebian Linux desktop OS. FIGS. 4-6 illustrate Android mobile OS 430,Android kernel 520, and Hydroid OS 660 that may be employed in OSarchitecture 300 in more detail, according to various embodiments.

As illustrated in FIG. 4, Android OS 430 includes a set of C/C++libraries in libraries layer 432 that are accessed through applicationframework layer 440. Libraries layer 432 includes the “bionic” system Clibrary 439 that was developed specifically for Android to be smallerand faster than the “glibc” Linux C-library. Libraries layer 432 alsoincludes inter-process communication (“IPC”) library 436, which includesthe base classes for the “Binder” IPC mechanism of the Android OS.Binder was developed specifically for Android to allow communicationbetween processes and services. Other libraries shown in libraries layer432 in FIG. 4 include media libraries 435 that support recording andplayback of media formats, surface manager 434 that managers access tothe display subsystem and composites graphic layers from multipleapplications, 2D and 3D graphics engines 438, and lightweight relationaldatabase engine 437. Other libraries that may be included in librarieslayer 432 but are not pictured in FIG. 4 include bitmap and vector fontrendering libraries, utilities libraries, browser tools (i.e., WebKit,etc.), and/or secure communication libraries (i.e., SSL, etc.).

Application framework layer 440 of Android OS 430 provides a developmentplatform that allows developers to use components of the devicehardware, access location information, run background services, setalarms, add notifications to the status bar, etc. Framework layer 440also allows applications to publish their capabilities and make use ofthe published capabilities of other applications. Components ofapplication framework layer 440 of Android mobile OS 430 includeactivity manager 441, resource manager 442, window manager 443, dockmanager 444, hardware and system services 445, desktop monitor service446, multi-display manager 447, and remote communication service 448.Other components that may be included in framework layer 440 of Androidmobile OS 430 include a view system, telephony manager, package manager,location manager, and/or notification manager, among other managers andservices.

Applications running on Android OS 430 run within the Dalvik virtualmachine 431 in the Android runtime environment 433 on top of the Androidobject-oriented application framework. Dalvik virtual machine 431 is aregister-based virtual machine, and runs a compact executable formatthat is designed to reduce memory usage and processing requirements.Applications running on Android OS 430 include home screen 451, emailapplication 452, phone application 453, browser application 454, and/orother application(s) (“App(s)”) 455.

The Android OS graphics system uses a client/server model. A surfacemanager (“SurfaceFlinger”) is the graphics server and applications arethe clients. SurfaceFlinger maintains a list of display ID's and keepstrack of assigning applications to display ID's. In one embodiment,mobile computing device 110 has multiple touch screen displays 116. Inthis embodiment, display ID 0 is associated with one of the touch screendisplays 116 and display ID 1 is associated with the other touch screendisplay 116. Display ID 2 is associated with both touch screen displays116 (i.e., the application is displayed on both displays at the sametime). Display ID's greater than 2 are virtual displays, meaning thatthey are not associated with a display physically present on mobilecomputing device hardware 112.

Graphics information for Android applications includes windows, views,and canvasses. Each window, view, and/or canvas is implemented with anunderlying surface object. Surface objects are double-buffered (frontand back buffers) and synchronized across processes for drawing.SurfaceFlinger maintains all surfaces in a shared memory pool whichallows all processes within Android to access and draw into them withoutexpensive copy operations and without using a server-side drawingprotocol such as X-Windows. Applications always draw into the backbuffer while SurfaceFlinger reads from the front buffer. SurfaceFlingercreates each surface object, maintains all surface objects, and alsomaintains a list of surface objects for each application. When theapplication finishes drawing in the back buffer, it posts an event toSurfaceFlinger, which swaps the back buffer to the front and queues thetask of rendering the surface information to the frame buffer.

SurfaceFlinger monitors all window change events. When one or morewindow change events occur, SurfaceFlinger renders the surfaceinformation to the frame buffer for one or more displays. Renderingincludes compositing the surfaces, i.e., composing the final image framebased on dimensions, transparency, z-order, and visibility of thesurfaces. Rendering may also include hardware acceleration (e.g., OpenGL2D and/or 3D interface for graphics processing hardware). SurfaceFlingerloops over all surface objects and renders their front buffers to theframe buffer in their Z order.

FIG. 5 illustrates modified Android kernel 520 in more detail, accordingto various embodiments. Modified Android kernel 520 includestouch-screen display driver 521, camera driver(s) 522, Bluetoothdriver(s) 523, shared memory allocator 524, IPC driver(s) 525, USBdriver(s) 526, WiFi driver(s) 527, I/O device driver(s) 528, and/orpower management module 530. I/O device driver(s) 528 includes devicedrivers for external I/O devices, including devices that may beconnected to mobile computing device 110 through port 120. ModifiedAndroid kernel 520 may include other drivers and functional blocksincluding a low memory killer, kernel debugger, logging capability,and/or other hardware device drivers.

FIG. 6 illustrates Hydroid OS 660 in more detail, according to variousembodiments. Hydroid is a full Linux OS that is capable of runningalmost any application developed for standard Linux distributions. Inparticular, libraries layer 662 of Hydroid OS 660 includes Linuxlibraries that support networking, graphics processing, databasemanagement, and other common program functions. For example, userlibraries 662 may include the “glibc” Linux C library 664, Linuxgraphics libraries 662 (e.g., GTK, OpenGL, etc.), Linux utilitieslibraries 661, Linux database libraries, and/or other Linux userlibraries. Applications run on Hydroid within an X-Windows Linuxgraphical environment using X-Server 674, window manager 673, and/ordesktop environment 672. Illustrated applications include word processor681, email application 682, spreadsheet application 683, browser 684,and other application(s) 685.

The Linux OS graphics system is based on the X-windows (or “X11”)graphics system. X-windows is a platform-independent, networked graphicsframework. X-windows uses a client/server model where the X-server isthe graphics server and applications are the clients. The X-servercontrols input/output hardware associated with the Linux OS such asdisplays, touch-screen displays, keyboards, pointing device(s), etc. Inthis regard, X-windows provides a server-side drawing graphicsarchitecture, i.e., the X-server maintains the content for drawablesincluding windows and pixmaps. X-clients communicate with the X-serverby exchanging data packets that describe drawing operations over acommunication channel. X-clients access the X communication protocolthrough a library of standard routines (the “Xlib”). For example, anX-client may send a request to the X-server to draw a rectangle in theclient window. The X-server sends input events to the X-clients, forexample, keyboard or pointing device input, and/or window movement orresizing. Input events are relative to client windows. For example, ifthe user clicks when the pointer is within a window, the X-server sendsa packet that includes the input event to the X-client associated withthe window that includes the action and positioning of the eventrelative to the window.

Because of the differences in operating system frameworks, graphicssystems, and/or libraries, applications written for Android do notgenerally run on Hydroid OS 660 and applications written for standardLinux distributions do not generally run on Android OS 430. In thisregard, applications for Android OS 430 and Hydroid OS 660 are notbytecode compatible, meaning compiled and executable programs for one donot run on the other.

In one embodiment, Hydroid OS 660 includes components of across-environment communication framework that facilitates communicationwith Android OS 430 through shared kernel 520. These components includeIPC library 663 that includes the base classes for the Binder IPCmechanism of the Android OS and remote communications service 671.

In one embodiment, Hydroid OS 660 is run within a chrooted (created withthe ‘chroot’ command) secondary execution environment created within theAndroid root environment. Processes and applications within Hydroid OS660 are run within the secondary execution environment such that theapparent root directory seen by these processes and applications is theroot directory of the secondary execution environment. In this way,Hydroid OS 660 can run programs written for standard Linux distributionswithout modification because Linux user libraries 662 are available toprocesses running on Hydroid OS 660 in the chrooted secondary executionenvironment.

Referring back to FIG. 3, mobile OS 130 and desktop 160 are in activeconcurrent execution on shared kernel 320 on a mobile device. Mobile OS130 and desktop OS 160 may be incompatible with regard to userlibraries, graphics systems, and/or application frameworks. This meansthat an application compiled for one OS (i.e., compiled against the userlibraries, etc., of that OS) will not execute on the other OS.Accordingly, mobile OS 130 and desktop OS 160 of OS architecture 300provide separate user experiences through different applicationsaccessible through separate user interaction spaces.

In some instances, an application may be available for mobile OS 130that provides similar functionality to an application available fordesktop OS 160. These applications, while separate applications compiledfor different OSs, may be compatible from the standpoint of user dataincluding document files. Examples of applications available on(compiled for and loaded within the execution environment of) both OSsmay include web browsers, email client applications, office productivityapplications (e.g., word processing, spreadsheet, presentation, etc.),photo and video editing applications, and/or other applications.Consequently, it may be possible for a user to edit a certain documentfile using a first application running on mobile OS 130, and later editthe same document file using a second application running on desktop OS160. However, this does not provide a seamless user experience. Toswitch between editing the document file with the first application andediting the document using the second application, the user must closethe document file with the first application, launch the secondapplication on the desktop OS, and re-open the document file with thesecond application. In this process, the user loses any applicationstate information related to their current state of use of the firstapplication not saved in the document file. This may include, forexample, the current editing or cursor position, the current view, undoinformation, the current editing mode (e.g., viewing with markup orwithout, etc.), currently selected text, selected menu tabs, applicationoptions (open toolbars, printing options, etc.).

Embodiments of the present invention extend the functionality of OSarchitecture 300 to provide a more seamless computing experience in amulti-OS computing environment by sharing application state informationacross applications executing on different OSs such that they provide asingle apparent application interaction state. This functionalityenables, for example, a user to seamlessly switch from interacting witha set of user data using a first application running in a first OS tointeracting with the same set of user data using a second applicationrunning in a second OS. Notably, the current user interaction state ismaintained to provide a seamless cross-environment workflow. This meansthat the user does not have to close application documents associatedwith the user data before switching to interacting with the user datausing the second application. Further, application interaction stateinformation that may not be part of an application document file ismaintained across the separate applications running on separate OSs. Forexample, user selected objects or text may be maintained acrossapplications running in separate OSs such that a user can begin a useraction on a set of user data with an application running in a first OS,and complete the action with an application running on a second,separate OS.

To provide a seamless cross-environment workflow, the embodimentsdescribed below provide methods and systems that allow two or moreapplication programs running in independent operating systems to shareuser interaction state information. The embodiments provide for sharinguser interaction state information across applications running in two ormore separate operating systems. Shared user interaction stateinformation may include user data, user settings, and/or applicationcontext information. In this way, two or more applications, compiled forand running on separate OSs, can present a single apparent userinteraction state.

FIG. 7 illustrates computing environment 300 a which extends computingenvironment 300 illustrated in FIG. 3 to provide seamless translation ofapplication data, user settings, and/or application context acrossapplications running in separate operating systems. In computingenvironment 300 a, applications 752 and 754 are compiled for and runningon mobile OS 130. Applications 782 and 784 are compiled for and runningin desktop OS 160, which runs concurrently with mobile OS 130 on sharedkernel 320. One or more applications available on mobile OS 130 mayinclude functionality for seamless translation of application data witha corresponding application available on desktop OS 160. For example,application 754 of mobile OS 130 may be a word processing applicationcompiled for mobile OS 130. Application 782 of desktop OS 160 may be acompatible (i.e., compatible document file types) word processingapplication compiled for desktop OS 160. Computing environment 300 aincludes functionality for providing seamless translation of a currentapplication interaction state including application data, applicationuser settings, and/or application context information across theapplications.

In computing environment 300 a, application state information is storedwithin application state data store 742 of mobile OS 130. Whileapplication state data store is illustrated as a component of mobile OS130, application state data store 742 may alternatively be a componentof desktop OS 160. Application state data store 742 is accessible byapplications within mobile OS 130 and applications within desktop OS160. In one embodiment, application state data store 742 is a databasemaintained by a service of mobile OS 130. Communication betweenapplications running on desktop OS 160 with the application state datastore is provided through an inter-process communication channel 725 ofshared kernel 320. A cross-environment communication framework thatprovides a communication channel for applications of desktop OS 160 tocommunicate with services and applications of mobile OS 130 is describedin more detail in co-pending U.S. patent application Ser. Nos.13/217,099, filed Aug. 24, 2011, entitled “Cross-EnvironmentCommunication Framework,” and 13/247,885, filed Sep. 28, 2011, entitled“Cross-Environment Communication Using Application Space API,” theentire contents of each of which are incorporated herein by reference.However, any suitable inter-process communication mechanism may be usedto provide access to application state data store by processes runningon desktop OS 160.

In the embodiment illustrated by FIG. 7, computing environment 300 aincludes application communication API 758 of mobile OS 130 andapplication communication API 788 of desktop OS 160. Applicationcommunication API 758 and application communication API 788 provide thelow-level functions for applications to communicate with applicationstate data store 742 to set and query application state information.Using these functions, mobile OS applications call down to applicationcommunication API 758 to set and query application state data. Forexample, application 754 calls functions of API 758 with name/valuepairs to set and query application state data. Desktop OS applicationscall down to application communication API 788 to set and queryapplication state data from application state data store 742.

The application communication APIs of computing environment 300 a may bein application space within the first and/or second OS. For example, API758 could be implemented as a runtime library of mobile OS 130. In oneembodiment, API 758 is implemented as a JAR file for a Dalvik runtimeenvironment of an Android mobile OS. In other embodiments, theapplication communication APIs of computing environment 300 a may bewithin user libraries or other libraries accessible to applications ofthe OS. For example, in one embodiment, API 788 of desktop OS 160 isimplemented as a shared user library in a Linux OS.

In alternative embodiments, computing environment does not include APIsin mobile OS 130 and desktop OS 160. In these embodiments, theapplications communicate directly with the application state data storeto set and query application state information. Applications of mobileOS 130 and desktop OS 160 may also communicate to the application statedata store through a service of mobile OS 130 or desktop OS 160.

FIGS. 8-10 illustrate how computing environment 300 a allows a seamlesscross-environment workflow, according to various embodiments. In a firstcomputing environment configuration 800 a illustrated in FIG. 8, a useris actively interacting with a set of information (e.g., spreadsheetdocument) using a first spreadsheet application (“CALCAPP”) running onmobile OS 130 on mobile device 110. Desktop OS 160 is runningconcurrently on the mobile device (on processor 114 of the mobile device110) with mobile OS 130 on shared kernel 320. The current userinteraction state of the first spreadsheet application includes usercontent, user settings, and an application context.

User content may include the data and formatting of the data that theuser is editing. User content may be stored in a document file 824stored on a non-volatile storage medium of the mobile computing device.A temporary document file 834 may also contain user content. In variousembodiments, temporary document file 834 includes user content that hasnot been saved to document file 824 by the user. For example, temporarydocument file 834 may include the changes made to the user documentsince it was last saved to document file 824 by the user.

It is worth noting that the document file 824 and/or the temporarydocument file 834 can alternatively be stored remote to the mobilecomputing device. For example, embodiments may store some or all of thedata in external storage (e.g., an external hard drive, a thumb drive,etc.), on a remote server accessible to the mobile computing device overa local or non-local network, on remote (e.g., “cloud”) storageaccessible to the mobile computing device over the public Internet, orin any other useful location. Other embodiments store some files locallyand others non-locally. Still other embodiments locally store copies(e.g., temporary versions, etc.) of non-local files and provideadditional functionality to synchronize those files (e.g., or keep trackof revisions, perform other file management tasks, etc.). Accordingly,embodiments described herein with reference to locally stored files areintended to be applicable to applications storing some or all of theirfiles in non-local storage. In some cases, handling of remote files isfacilitated by client-server interactions, or the like, as discussedmore fully below.

User settings include user selected options, such as, for example,default document formatting and views, visible menus, hotkeys, documentreview options, proofing options, and/or user customizations. Typically,document specific settings and options are stored in the document filewhile other options may be stored in a file associated with theapplication program. In computing environment 300 a, options that aretypically stored in the file associated with the application program maybe stored in application state data store 742.

The application context includes additional information that defines thecurrent user interaction state of the application. For example,application context information not typically stored in a document oruser settings file may include the current cursor position, selectedtext or objects, previous user actions (e.g., undo levels and actions),current view, selected menu tab, and/or other information. In computingenvironment configuration 800 a, application context informationincludes cursor position 855 and selected cell 856. This applicationcontext information is maintained in application state data store 742,and is represented in computing environment configuration 800 a byapplication state 844. In addition, the user has entered data since lastsaving document file 824. For example, the user may have entered aformula in computing environment configuration 800 a since last savingthe user document. This information is stored in temporary document file834.

After some time period of interacting with the set of information usingthe first spreadsheet application on the mobile device, the user docksthe mobile device with a second user environment (e.g., desktop userenvironment). For example, FIG. 9 illustrates computing environmentconfiguration 800 b in which mobile computing device 110 is docked withsecondary terminal environment 940, according to various embodiments.Secondary terminal environment 940 includes display 944, keyboard 946,and pointing device 948. In this regard, secondary terminal environment940 is a desktop-like terminal environment. When docked through dockinterface 922, the mobile device associates desktop OS 160 withsecondary terminal environment 940 such that the user interface 980 ofthe desktop OS is displayed on the display 944 of the second userenvironment 940. As illustrated in FIG. 9, dock interface 922 mayinclude a dock cradle 941 and/or dock cable 943.

In this instance, the user wishes to continue to interact with the sameuser content (i.e., spreadsheet data represented by file ‘File.ods’)using an application that is tailored to the context of the second userenvironment, without losing the current user interaction state. Forexample, while compatible from the standpoint of document file types,applications available for the mobile OS may differ in some respects toapplications available for the desktop OS. These differences may includethe screen resolution, menus, visible document area, functionality, etc.For these reasons, it may be easier for the user to edit user contentusing the desktop OS application when the mobile computing device isdocked to a secondary terminal environment.

FIG. 10 illustrates computing environment configuration 800 c that showshow aspects of computing environment 700 provide a seamless workflowacross multiple applications running on separate operating systems,according to various embodiments. When the user docks mobile computingdevice 110 to secondary terminal environment 940, the first spreadsheetapplication 854 closes document file 824 and temporary document file 834and shuts down or suspends (no longer uses computing resources of mobileOS 130) as illustrated in FIG. 10 by a dashed outline. Application 1084launches on desktop OS 160 and accesses application state data store 742to retrieve the current user interaction state of application 854 (i.e.,the user interaction state of application 854 just before mobilecomputing device 110 was docked with secondary terminal environment940).

As described above, the user interaction state information 844 includesuser content, user settings, and application context information. Userinteraction state information 844 includes pointers to document file 824and temporary file 834 that contain the user content information. Thesecond spreadsheet application opens the document file 824 and temporarydocument file 834, and queries the application state information 844 foruser settings and application context information. For example, thesecond application retrieves the cursor position 855 and active cell856, among other application context information. Accordingly, thesecond spreadsheet application 1084 presents an application window 1086on the secondary terminal environment that allows the user to seamlesslypick up interacting with spreadsheet user data exactly where they leftoff with the first application 854. Using a similar process, the usermay switch back to editing user data using the first spreadsheetapplication 854 running on the mobile OS when the user undocks themobile computing device.

It is not necessary for a first application running on a first OS tohave exactly the same user interface as a second application running ona second OS to present a single apparent user interaction state usingthe above embodiments. For example, the user interface style, way ofinteracting with the user interface, and/or user interface componentsmay be different while still considered to be presenting a singleapparent user interaction state. In particular, each application may betailored to the context of the operating system on which it is runningas well as the user environment on which it is displayed, while sharinga single apparent user interaction state. For example, mobileapplications may be tailored to the touch-based user input and smallerdisplay screen present on typical mobile computing environments. Incontrast, desktop applications may be tailored to larger screens andpointing device(s) based input. For example, a desktop word-processingapplication may have more visible menus and menu icons that does aword-processing application designed for a mobile OS GUI. However, usingthe embodiments described above, application context information may beshared to provide a single apparent user interaction state thatprovides, to the user, a seamless workflow for editing user contentusing either the mobile application or the desktop application.

In the embodiment described above, switching between applicationsrunning on separate operating systems and sharing application stateinformation occurred automatically based on an event (e.g., docking,undocking, etc.). That is, the computing environment assumed that theuser intends to switch to the desktop application when the user docksthe mobile device. However, switching between applications with whichthe user is actively interacting may occur as a result of a user prompt,or based on user settings. For example, when the user docks the mobiledevice, the user could be prompted to determine whether they want tocontinue to interact with the first application running on the mobileOS, or switch to interacting with the same user data through a secondapplication running on the desktop OS. Alternatively, user settingscould determine that for some applications the user wants to switchautomatically when the mobile device is docked, while for otherapplications the user either wants to be prompted or does not want toswitch to the desktop application.

In embodiments, applications running on separate OSs and sharing asingle apparent user interaction state may both be running at the sametime. For example, application 854, running on mobile OS 130 may berunning and presenting a user application window through the userenvironment associated with mobile OS 130. At the same time, application1084 is running on desktop OS 160 and presenting application window 1086on a second user environment associated with desktop OS 160. In thisinstance, applications 854 and 1084 may share user settings andapplication context information as described above. When applicationcontext information is changed (i.e., the user interaction state isupdated) in response to a user input to one of applications 854 and1084, the application context information is updated in applicationstate data store 742. Application state data store 742 pushes the changeto the other application through a notification channel such that theother application maintains a consistent user interaction state with thefirst application. For concurrently running applications on separateoperating systems, releasing locks on document file 824 and temporarydocument file 834 may be done in several ways. In one embodiment,document files are released or established in response to user input inone of the concurrently running applications.

FIG. 11 illustrates a process flow 1100 for seamless translation ofapplication data between separate applications displayed in separateuser environments (e.g., running on independent operating systems, eachoperating system associated with a separate user environment), accordingto various embodiments. Process flow 1100 begins at block 1102, where afirst application running on a first operating system opens a documentfile, the user content of the document file representing a storedocuments state. At block 1104, the first application accepts userinput, and, as a result of this user input, has a first user interactionstate. In this first user interaction state, the document has a modifieddocument state. That is, the user document has been changed by the userto become a modified user document. At block 1106, application stateinformation representing the first user interaction state is storedwithin the first computing environment, the application stateinformation including application context information. At block 1108,the first application releases a file lock of the document file withoutsaving the modified document state to the document file. At block 1110,a second application running on a second operating system opens thedocument file. At block 1112, the second application accesses theapplication state information associated with the first user interactionstate. At block 1114, the second application applies the applicationcontext information to present an interaction state of the secondapplication that is substantially similar to the first user interactionstate. As a result of applying the application context information, themodified document has been transitioned from the first application,running on a first OS, to the second application, running on a secondOS, with the application context maintained. That is, the user does notlose their current working application state, without having to save,close, and reopen the file, and without losing application contextinformation not saved in the document file.

Seamless translation of application data between applications running onseparate operating systems presents different issues for different typesof applications. The description provided above shows how a seamlessworkflow may be provided across applications running on multipleseparate operating systems for a category of applications that are usedprimarily to create and modify local user content. This categoryincludes, for example, word processing applications, spreadsheetapplications, presentation applications, and photo and video editingapplications, among others.

Another category of applications includes applications that include, asa primary aspect, communication with a remote server. These applicationsmay be said to be client-server type applications. For example, thiscategory can include client-server communication applications, such ase-mail client applications, web browser applications, cloud-based or“software-as-a-service” applications, and the like. These applicationsmay store user data and settings differently than applications thatprimarily operate on local user content. For example, an email clientapplication may connect to an email server that provides storage foremail folders. The email server may maintain folders of most usercontent and allow multiple email clients on multiple platforms toconnect to the server to interact with the user content. The server maypush received emails to connected email clients.

Accordingly, client-server type applications may store a more limitedset of information locally than do applications that operate primarilyon local user content. It will be appreciated, however, that it isbecoming increasingly common for applications to store some informationlocally and other information non-locally. As such, the categorizationof applications according to their type of data storage is intended onlyto add clarity to the discussion of functionality of various embodimentsand is not intended to limit the scope of any embodiment as fallingwithin only a single category of functionality. For example, an emailclient may act more like a local storage type of application in certainuse cases (e.g., when composing a locally-saved draft of an email usinga non-web-based email client), while the same email client may act morelike a client-server application in other use cases (e.g., whenexchanging email data with a remote email server). Indeed, when anapplication is (or is acting more like) a client-server typeapplication, the application may need to provide functionality that maynot be needed in an application having purely local interactions. Forexample, for a particular client-server application, multiple clientsmay each need to send and receive information from the server whilemaintaining a single, substantially unified application interactionstate.

In one embodiment for providing seamless translation of applicationstate between multiple client applications in a client-server context, afirst client application (e.g., email client), running on a firstoperating system (e.g., mobile OS 130), communicates with a remotecommunications server over a communications protocol (e.g., IMAP, POP3,Exchange, etc.). The first client stores all application stateinformation in an application state data store on the first operatingsystem. For example, an email client application may store user data(e.g., email folders, draft email, calendar appointments, etc.), usersettings (e.g., screen layout, message appearance, notificationsettings, etc.), and application context information (e.g., currentcursor position, highlighted text, undo information and levels, etc.).As a result of user interaction with the first client application, thefirst client application has a current user interaction state.

In response to an event (e.g., dock event), a second client applicationlaunches on a second operating system and connects to the applicationstate data store. The second application receives the application stateinformation for the current user interaction state of the firstapplication including user content, user settings, and applicationcontext information from the application state data store. That is, thesecond client application does not directly communicate with the server.Instead, the first client application performs all communication withthe server and maintains a local copy of all application state data inthe application state data store. The second client application receivesapplication state information from the application state data store.

In an alternative embodiment for providing seamless translation ofapplication state between multiple client applications in aclient-server context, a first client application (e.g., email client),running on a first operating system (e.g., mobile OS 130), communicateswith a remote communications server over a communications protocol(e.g., IMAP, POP3, Exchange, etc.). The first client stores applicationstate information in an application state data store. The applicationstate information includes user content, user settings, and applicationcontext information that is not typically stored on the remote server.For example, an email client application may store user settings (e.g.,screen layout, message appearance, notification settings, etc.), andapplication context information (e.g., current cursor position,highlighted text, undo information and levels, etc.). As a result ofuser interaction with the first client application, the first clientapplication has a current user interaction state.

In response to an event (e.g., dock event), a second client applicationlaunches on a second operating system, connects to the remote server,and connects to the application state data store to receive theapplication state information. For example, an email client applicationrunning on the second OS may connect to the server for most user content(e.g., email folders, draft email, calendar appointments, etc.), andconnect to the application state data store for additional applicationstate information including user settings (e.g., screen layout, messageappearance, notification settings, etc.) and application contextinformation (e.g., current cursor position, highlighted text, undoinformation and levels, etc.). The second application rebuilds thecurrent user interaction state of the first application from the usercontent received from the remote server and the application stateinformation received from the application state data store.

Some other types of client-server applications provide an interface andfunctionality to navigate and present Internet-based content. Thesetypes of applications may include, for example, web browsers andcloud-based applications (e.g., productivity applications, filemanagement and/or storage applications, streaming media applications,Internet radio applications, IP-based telephony and/or conferencingapplications, etc.). Preserving an apparently common applicationinteraction state across these types of applications running in multipleuser environments can present different and/or additional issues fromthose situations involving purely local applications. For example, webbrowsers can maintain a substantial amount of information relating touser settings and sessions of the browser. In addition, web content canbe highly dynamic and interactive (e.g., CSS, flash animation, applets,javascript, forms, etc.). A web-browser may also run applications withinthe browser (e.g., applets, etc.), have varying access to or support forplug-ins, etc. In some instances, the web-browser may be used as avirtual operating system within an operating system (e.g., applets,cloud-based applications). Further, web browsers may be used to providesecure sessions and/or other secure communications, user authenticationand/or tracking, and the like.

Returning to FIGS. 8-10 for the sake of illustration, a mobile browseris running as application 854 on mobile OS 130 and is presenting a userapplication window through the user environment associated with mobileOS 130. Desktop OS 160 is running concurrently on the mobile device (onprocessor 114 of the mobile device 110) with mobile OS 130 on sharedkernel 320. The current user interaction state of the mobile browserapplication includes user content, user settings, and an applicationcontext. User content for the mobile browser application may includecookies, tokens, user data (e.g., being provided to a form, etc.),and/or any other information generated for or by the user in relation tothe browser (e.g., including any browser-implemented applications orsessions). User settings may include home page settings, saved browserhistory, saved bookmarks or favorites, proxy settings, and/or any otherinformation stored for the user in relation to the browser. Applicationcontext may include current state information for the browser. In someembodiments, application context only includes the currently displayedURL. In other embodiments, the application context includes additionalinformation, such as currently selected text, current pointer location,etc.

At some time, the user docks mobile computing device 110 to secondaryterminal environment 940 (e.g., as illustrated in FIG. 9). It may bedesirable to switch the user's interaction context from the mobilebrowser to a desktop browser application 1084 running on desktop OS 160and presenting application window 1086 on a second user environmentassociated with desktop OS 160 (e.g., as illustrated in FIG. 10). Mobilebrowser application 854 and desktop browser application 1084 may shareuser settings and application context information as described above. Insome cases, the shared application context involves passing only thecurrent URL; while in other cases, more advanced context sharing occurs,as described more fully below. As discussed above, application contextinformation can be updated in the application state data store 742, andthe application state data store 742 can push the change to the otherapplication through a notification channel to maintain a consistent userinteraction state, where desired.

A number of browser-specific issues may be handled in different ways bydifferent embodiments. One such issue is that many web servers areconfigured to deliver different content (or the same content displayedin a different format) for different types of browsers. This can beparticularly true for mobile versus desktop types of browsers, as manyweb pages are designed to exploit the different user interactionenvironments differently (e.g., to exploit differences in display size,resolution, support for plug-ins like Flash, etc.). An HTML request fora webpage from a browser may include an “agent string” that identifiesbrowser characteristics, including browser type. A web server canrespond to the request differently depending on whether the agent stringindicates that the request came from a mobile versus a desktop browser.Suppose a user is viewing a mobile version of a webpage on the mobiledevice through the mobile browser application, and the user then dockswith the desktop environment.

In one scenario, the mobile version of the webpage is displayed on thedesktop environment as a remotely rendered window of the mobile browser.For example, techniques for remote rendering in this way are describedin U.S. patent application Ser. No. 13/246,665, filed Sep. 27, 2011,entitled “Instant Remote Rendering,” the entire contents of which isincorporated herein by reference. This type of remote rendering may beimplemented as a default condition for all pages being viewed prior to adocking event. Alternatively, the remote rendering may be invoked aftera determination is made that the mobile version of the webpage would notrender (e.g., that requesting the mobile version of the webpage from thedesktop browser would result in an error) or that the webpage wouldrender improperly or in an undesirable way.

In another scenario, the URL of the mobile version of the webpage istransferred to the desktop browser as context information. The desktopbrowser can then request the URL again and/or use any cached information(e.g., which may be stored as user data or context information).Depending on the implementation of the webpage, a number of results arepossible. For example, the desktop may be able to display theinformation returned from the web server, the web server may return anerror indicating that the browser is incompatible with the request, theweb server may reinterpret the request and return the desktop version ofthe requested website, etc. Alternatively, the desktop browser may beconfigured to change the agent string in the request to appear as amobile browser.

Another such browser-related issue can occur when the user isinteracting with content over an SSL session, or the like. Someembodiments may simply not allow transfer of this session to the desktopenvironment (e.g., to the desktop browser), and may, for example, returnan error, warning, or other indication. Other embodiments establish theSSL session through a proxy server, which may effectively route the SSLsession to the appropriate mobile or desktop browser. For example, theproxy server may be configured to act as a network address translator(NAT), which handles the SSL session and routes the sessioncommunications (e.g., securely) to one or more of the mobile or desktopbrowsers. Still other embodiments remotely render the SSL session fromthe mobile browser to the desktop environment, as discussed above. Anyof these embodiments may further include detecting the SSL session priorto performing functionality to handle the SSL session.

Yet another such browser-related issue can occur when the user isinteracting with a webpage that involves cookies, or the like, prior tothe docking event. According to some embodiments, the cookie informationeither transfers to the desktop browser environment upon docking or isstored as context information that is accessible to the desktop browser.Certain types of cookies and the like cannot transfer (or at leastcannot transfer in a way that is reliable or useful). Accordingly, someembodiments handle this type of cookie information either by ignoringit, by translated it to transferable information where possible, etc.Similar scenarios may be handled by various embodiments in the contextof user authentication information, tokens, etc. For example, in thecase of a webpage to which the user has logged in, after docking, thedesktop browser may return the user to a login page or display the pagewithout incident, depending on the implementation of the webpage'ssource code, etc. Alternatively, that type of user authentication may bedetected, and the docking event may cause the transfer of theinteraction environment to be handled otherwise (e.g., by remoterendering, by proxy server handling, etc.).

Still another such browser-related issue can occur when the user isinteracting with web-based content that involves user data. Depending onthe implementation of the content, the user data may be stored locallyand/or non-locally. For example, a web-based productivity application(e.g., word processor, spreadsheet application, etc.), game, etc. mayinclude a current user state, selected text, settings, etc. Embodimentstypically handle locally stored data as discussed above. Regardingnon-local data, some embodiments either re-request the data for the newbrowser environment when needed after docking or ignore the data when itcannot practically be re-requested.

Even another such browser-related issue can occur when the user isinteracting with (e.g., actively interacting or just has open) multiplewebpages concurrently. Typically, mobile browsers are configured todisplay only a single webpage (or a very limited number of webpages) ata time, while desktop browsers can use multiple browser windows and/ormultiple tabs to display many webpages at a time. Some embodimentstransfer each open webpage in the mobile browser to a new tab or windowin the desktop browser (e.g., according to user settings, defaultsettings, etc.). As discussed above, a number of cases can arise inwhich the webpage cannot simply be transferred (or cannot be transferredin a desirable way) from the mobile environment to the desktopenvironment. According to certain embodiments, each webpage to betransferred is treated independently. For example, for each webpage thatcannot be simply transferred, the webpage is either not transferred(e.g., kept on the mobile environment only), displayed through remoterendering, translated to an appropriate desktop version, routed througha proxy, etc.

FIG. 12 illustrates a process flow 1200 for seamless translation ofapplication state between multiple client applications in aclient-server context, according to various embodiments. Process flow1200 begins at block 1202, where user settings and application contextinformation are maintained in a first computing environment in relationto a first client/server application running on a first operating systemof the first computing environment, where the first computingenvironment is configured to run the first operating system and a secondoperating system concurrently on a shared kernel. At block 1204, a firstuser interaction environment of the first client-server application isdisplayed via the first computing environment according to the usersettings and the application context information, the first userinteraction environment reflecting a current interaction state. At block1206, a docking event is detected indicating that the mobile computingenvironment is docked with a desktop computing environment. At block1208, a second client-server application running in the second operatingsystem applies the application context information to present a seconduser interaction environment via the second client-server application ona display of the second computing environment that substantiallyreflects the current interaction state. Through process flow 1200, theuser can switch seamlessly from the first client/server application inthe first computing environment to the second client/server applicationin the second computing environment without losing the currentinteraction state.

It will be appreciated that the above issues and scenarios are intendedonly to be illustrative of possible cases and techniques for handlingthose cases. Many other similar cases are possible and can be handledaccording to the techniques described above. Further, descriptionsreferring specifically to webpages can, in most cases, also be handledsimilarly for other web-based content, such as web-based (e.g.,cloud-based, software-as-a-service, etc.) games and applications. Assuch the specific descriptions above of client-server applicationsshould not be construed as limiting the scope of embodiments describedherein.

The foregoing description has been presented for purposes ofillustration and description. Furthermore, the description is notintended to limit embodiments of the invention to the form disclosedherein. While a number of exemplary aspects and embodiments have beendiscussed above, those of skill in the art will recognize certainvariations, modifications, permutations, additions, and sub-combinationsthereof.

The various operations of methods described above may be performed byany suitable means capable of performing the corresponding functions.The means may include various hardware and/or software component(s)and/or module(s), including, but not limited to a circuit, anapplication specific integrated circuit (ASIC), or processor.

The various illustrative logical blocks, modules, and circuits describedmay be implemented or performed with a general purpose processor, adigital signal processor (DSP), an ASIC, a field programmable gate arraysignal (FPGA), or other programmable logic device (PLD), discrete gate,or transistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. A generalpurpose processor may be a microprocessor, but in the alternative, theprocessor may be any commercially available processor, controller,microcontroller, or state machine. A processor may also be implementedas a combination of computing devices, e.g., a combination of a DSP anda microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

The steps of a method or algorithm described in connection with thepresent disclosure, may be embodied directly in hardware, in a softwaremodule executed by a processor, or in a combination of the two. Asoftware module may reside in any form of tangible storage medium. Someexamples of storage media that may be used include random access memory(RAM), read only memory (ROM), flash memory, EPROM memory, EEPROMmemory, registers, a hard disk, a removable disk, a CD-ROM and so forth.A storage medium may be coupled to a processor such that the processorcan read information from, and write information to, the storage medium.In the alternative, the storage medium may be integral to the processor.A software module may be a single instruction, or many instructions, andmay be distributed over several different code segments, among differentprograms, and across multiple storage media.

The methods disclosed herein comprise one or more actions for achievingthe described method. The method and/or actions may be interchanged withone another without departing from the scope of the claims. In otherwords, unless a specific order of actions is specified, the order and/oruse of specific actions may be modified without departing from the scopeof the claims.

The functions described may be implemented in hardware, software,firmware, or any combination thereof. If implemented in software, thefunctions may be stored as one or more instructions on a tangiblecomputer-readable medium. A storage medium may be any available tangiblemedium that can be accessed by a computer. By way of example, and notlimitation, such computer-readable media can comprise RAM, ROM, EEPROM,CD-ROM, or other optical disk storage, magnetic disk storage, or othermagnetic storage devices, or any other tangible medium that can be usedto carry or store desired program code in the form of instructions ordata structures and that can be accessed by a computer. Disk and disc,as used herein, include compact disc (CD), laser disc, optical disc,digital versatile disc (DVD), floppy disk, and Blu-ray®disc where disksusually reproduce data magnetically, while discs reproduce dataoptically with lasers.

Thus, a computer program product may perform operations presentedherein. For example, such a computer program product may be a computerreadable tangible medium having instructions tangibly stored (and/orencoded) thereon, the instructions being executable by one or moreprocessors to perform the operations described herein. The computerprogram product may include packaging material.

Software or instructions may also be transmitted over a transmissionmedium. For example, software may be transmitted from a website, server,or other remote source using a transmission medium such as a coaxialcable, fiber optic cable, twisted pair, digital subscriber line (DSL),or wireless technology such as infrared, radio, or microwave.

Further, modules and/or other appropriate means for performing themethods and techniques described herein can be downloaded and/orotherwise obtained by a user terminal and/or base station as applicable.For example, such a device can be coupled to a server to facilitate thetransfer of means for performing the methods described herein.Alternatively, various methods described herein can be provided viastorage means (e.g., RAM, ROM, a physical storage medium such as a CD orfloppy disk, etc.), such that a user terminal and/or base station canobtain the various methods upon coupling or providing the storage meansto the device. Moreover, any other suitable technique for providing themethods and techniques described herein to a device can be utilized.

Other examples and implementations are within the scope and spirit ofthe disclosure and appended claims. For example, due to the nature ofsoftware, functions described above can be implemented using softwareexecuted by a processor, hardware, firmware, hardwiring, or combinationsof any of these. Features implementing functions may also be physicallylocated at various positions, including being distributed such thatportions of functions are implemented at different physical locations.Also, as used herein, including in the claims, “or” as used in a list ofitems prefaced by “at least one of” indicates a disjunctive list suchthat, for example, a list of “at least one of A, B, or C” means A or Bor C or AB or AC or BC or ABC (i.e., A and B and C). Further, the term“exemplary” does not mean that the described example is preferred orbetter than other examples.

Various changes, substitutions, and alterations to the techniquesdescribed herein can be made without departing from the technology ofthe teachings as defined by the appended claims. Moreover, the scope ofthe disclosure and claims is not limited to the particular aspects ofthe process, machine, manufacture, composition of matter, means,methods, and actions described above. Processes, machines, manufacture,compositions of matter, means, methods, or actions, presently existingor later to be developed, that perform substantially the same functionor achieve substantially the same result as the corresponding aspectsdescribed herein may be utilized. Accordingly, the appended claimsinclude within their scope such processes, machines, manufacture,compositions of matter, means, methods, or actions.

1. A method comprising: opening a document file with a first applicationrunning on a first operating system, the user content of the documentfile representing a stored document state; accepting user input in thefirst application, the first application having, as a result ofresponding to the user input, a first user interaction state, the firstuser interaction state including a modified document state; storingapplication state information representing the first user interactionstate within the first computing environment, the application stateinformation including application context information; releasing a filelock of the document file by the first application without saving themodified document state to the document file; opening, by a secondapplication running on a second operating system, the document file;accessing, by the second application, the application state informationassociated with the first user interaction state; and applying, by thesecond application, the application context information to present aninteraction state of the second application that is substantiallysimilar to the first user interaction state.
 2. The method of claim 1,wherein the application context information includes a hint to atemporary file that represents changes between the stored document stateand the modified document state.
 3. The method of claim 1, furthercomprising opening, by the second application, a temporary document filecreated by the first application, the temporary document filerepresenting changes between the stored document state and the modifieddocument state.
 4. The method of claim 1, wherein the first applicationis displayed within a first user environment and the second applicationis displayed within second user environment.
 5. The method of claim 1,wherein the first and second operating systems run concurrently on ashared kernel of a mobile device.
 6. The method of claim 1, wherein theapplication context information includes at least one user interactionstate parameter not represented in the document file.
 7. The method ofclaim 1, wherein the application context information includesinformation associated with a non-completed user action.
 8. The methodof claim 1, wherein the application context information includesinformation associated with a current user selection of user contentinformation.
 9. The method of claim 1, wherein the application contextinformation is stored in a database accessed by the first and secondapplications through an application programming interface.
 10. Themethod of claim 1, wherein the opening of the document file by thesecond application is in response to detecting a docking event of amobile device.
 11. A computing device including a computer-readablemedium storing instructions for a physical processor, the instructions,when executed, causing the processor to perform steps comprising:opening a document file with a first application running on a firstoperating system, the user content of the document file representing astored document state; accepting user input in the first application,the first application having, as a result of responding to the userinput, a first user interaction state, the first user interaction stateincluding a modified document state; storing application stateinformation representing the first user interaction state within thefirst computing environment, the application state information includingapplication context information; releasing a file lock of the documentfile by the first application without saving the modified document stateto the document file; opening, by a second application running on asecond operating system, the document file; accessing, by the secondapplication, the application state information associated with the firstuser interaction state; and applying, by the second application, theapplication context information to present an interaction state of thesecond application that is substantially similar to the first userinteraction state
 12. The computing device of claim 11, wherein theapplication context information includes a hint to a temporary file thatrepresents changes between the stored document state and the modifieddocument state.
 13. The computing device of claim 11, wherein theinstructions further cause the processor to perform the step of opening,by the second application, a temporary document file created by thefirst application, the temporary document file representing changesbetween the stored document state and the modified document state. 14.The computing device of claim 11, wherein the first application isdisplayed within a first user environment and the second application isdisplayed within second user environment.
 15. The computing device ofclaim 11, wherein the first and second operating systems runconcurrently on a shared kernel of a mobile device.
 16. The computingdevice of claim 11, wherein the application context information includesat least one user interaction state parameter not represented in thedocument file.
 17. The computing device of claim 11, wherein theapplication context information includes information associated with anon-completed user action.
 18. The computing device of claim 11, whereinthe application context information includes information associated witha current user selection of user content information.
 19. The computingdevice of claim 11, wherein the application context information isstored in a database accessed by the first and second applicationsthrough an application programming interface.
 20. The computing deviceof claim 11, wherein the opening of the document file by the secondapplication is in response to detecting a docking event of the computingdevice.